A previous change (cockroachdb#49721) adding a benchmark for ExportToSst
left in some extra unused variables in a helper method. This
change cleans that up.

Release note: None.

This commit extract most of the logic of `createTableReaders` into
a separate method that performs the actual physical planning of table
readers given already created `TableReaderSpec`s. This will allow for
reuse in the follow-up work. This commit additionally slightly refactors
a few other things for reuse.

Release note: None

This commit extracts the logic of `execFactory.ConstructPlan` into the
function to be used by both factories. It also extracts out another
small helper.

Release note: None

Also fixed a bug where `\0` didn't actually output a NULL terminator (no
idea why it worked).

Release note (sql change): Populate the spatial_ref_sys table with
support SRID entries for geospatial data types.

This commit is automatic replacement of `apd.Decimal.SetFinite((.*), 0)`
into a nicer `apd.Decimal.SetInt64($1)` which are equivalent.

Release note: None

Trying to make the geo package not depend on GEOS, and this is one of
the required steps.

Release note: None

This was making it hard to run the same benchmarks in a consistent way.

Release note: None

The .eg.go files had unnecessary imports of execgen, which is not
required in the runtime generated code, and in fact is suspicious.

Release note: None

This commit adds a new directive to execgen:

// execgen:inline

This directive causes a function that's annotated with it to be inlined
into all of its callers via AST transformation. This kind of inlining is
smarter than before - it permits arguments with names different from the
names of the function's formal parameters, for example.

This commit also changes the functions in distinct_tmpl to use this
directive instead of the manual templating method.

I think this is the only easy function to change. The rest have, at the
simple level of difficulty, static template-time parameters, and at a
harder level of difficulty, type parameters. Improving these cases
holistically should come next.

Release note: None

…e a sentinel ErrFileDoesNotExist error

The `ReadFile` interface method is responsible for returning a `Reader`
which can be used to stream data from an external storage source.
There are also instances where we use this method to solely check for
the existence (or absence) of a particular file/object, by attempting
to open a stream. eg: checking for BackupManifest/BackupManifestCheckpoint
before exporting data. Previously, we would treat any error returned
by the storage vendor API as a signal for the file/object not existing.

This change adds logic to catch the native "file does not exist"
errors for each storage provider, and throw a sentinel error to users
of the `ReadFile` method. This allows for more careful error handling.

Relevant unit tests have also been added.

Release note: None

There's at lease one PR with a missing branch tips in refs/pull.
This commit makes the script tolerant of that failure.

Release note: None

49852: storage: Remove extra vars from runExportToSst r=itsbilal a=itsbilal

A previous change (cockroachdb#49721) adding a benchmark for ExportToSst
left in some extra unused variables in a helper method. This
change cleans that up.

Release note: None.

Co-authored-by: Bilal Akhtar <bilal@cockroachlabs.com>

Resume verb is already used to for the action of starting a job
after adoption on a node. Here instead Resume is the opposite
action of pausing a job.

Release note: none.

Fixes cockroachdb#49809.

This PR disallows using types from other databases in tables. This makes
certain behavior (like `DROP TYPE`) more predictable in their effects,
as well as unblocking some work for supporting user defined types in
`cockroach dump`.

Release note (sql change): Referencing types across databases has been
disabled.

As docgen relies on builtins which relies on proj, we are stuck in a
situation in cross compilation where we are trying to install docgen
with a non-native libproj.a file causing the linker to fail in `Upload
Binaries`. Fix this by not compiling docgen on cross compilations.

Also fix the windows compilation as PROJ will dump it into a
libproj_4_9.a instead of libproj.a. Cannot see an option to fix this
on the CMakeLists.txt.

Also make execgen not depend on LIBPROJ.

Release note: None

This commit adds an implementation of
`distSQLSpecExecFactory.ConstructScan` which combines the logic that
performs `scanNode` creation of `execFactory.ConstructScan` and physical
planning of table readers of `DistSQLPlanner.createTableReaders`.
I tried to refactor the code so that there is not much duplication going
on.

Notably, simple projections, renders, and filters are not yet
implemented.

Release note: None

After the examining the code closer, I see that now in all code paths we
populate `scanColumnsConfig.wantedColumns` to ask for the specific
columns to be scanned. Previously, this map could be left as `nil` which
would mean that all columns of the table should be scanned. This was
used only by the heuristic planner which has been removed, so we can now
update the assumption of the scan columns config. This allows us to
clean up the comments a bit.

This commit also moves `makeColumnsConfig` function into the shared util
file because it is used by both factories.

Release note: None

Prior to this commit, a recursive CTE in which the cardinality of
the left side of the UNION ALL expression was zero would cause an
error in the statistics code, "estimated row count must be non-zero".
This was happening because the cardinality of the recursive CTE binding
props was set to be non-zero, but the row count, which came from the
left side expression, was not updated accordingly. The stats code only
allows the row count to be zero if the cardinality is also zero.

This commit fixes the problem by setting the row count of the binding
props to 1 if the estimated row count of the left side is less than 1.

Fixes cockroachdb#49911

Release note (bug fix): Fixed an internal planning error that occured
for recursive CTEs (WITH RECURSIVE expressions) in which the left side
of the UNION ALL query used in the CTE definition produced zero rows.

A message had an unescaped format and ended up rendered like:
use YourFuncf("descriptive prefix %!s(MISSING)", ...)

Release note: None

49913: cloud: Added support to all ExternalStorage ReadFile methods, to raise a sentinel ErrFileDoesNotExist error r=adityamaru a=adityamaru

The `ReadFile` interface method is responsible for returning a `Reader`
which can be used to stream data from an external storage source.
There are also instances where we use this method to solely check for
the existence (or absence) of a particular file/object, by attempting
to open a stream. eg: checking for BackupManifest/BackupManifestCheckpoint
before exporting data. Previously, we would treat any error returned
by the storage vendor API as a signal for the file/object not existing.

This change adds logic to catch the native "file does not exist"
errors for each storage provider, and throw a sentinel error to users
of the `ReadFile` method. This allows for more careful error handling.

Relevant unit tests have also been added.

Release note: None

Co-authored-by: Aditya Maru <adityamaru@gmail.com>

Incorporate volatility of operators (unary, binary, comparison) into the
VolatilitySet property.

Unfortunately, this modifies most plans as pretty much everything is
"immutable". Perhaps once this work is behind us we will want to hide this
information from plans in most cases.

Release note: None

…db#49958 cockroachdb#49961

49728: execgen: add generic inliner r=jordanlewis a=jordanlewis

This commit adds a new directive to execgen:

// execgen:inline

This directive causes a function that's annotated with it to be inlined
into all of its callers via AST transformation. This kind of inlining is
smarter than before - it permits arguments with names different from the
names of the function's formal parameters, for example.

This commit also changes the functions in distinct_tmpl to use this
directive instead of the manual templating method.

I think this is the only easy function to change. The rest have, at the
simple level of difficulty, static template-time parameters, and at a
harder level of difficulty, type parameters. Improving these cases
holistically should come next.

Release note: None

49841: sql: disallow cross database type references r=otan a=rohany

Fixes cockroachdb#49809.

This PR disallows using types from other databases in tables. This makes
certain behavior (like `DROP TYPE`) more predictable in their effects,
as well as unblocking some work for supporting user defined types in
`cockroach dump`.

Release note (sql change): Referencing types across databases has been
disabled.

49919: sql: use nicer apd.Decimal.SetInt64 in the code base r=yuzefovich a=yuzefovich

This commit is automatic replacement of `apd.Decimal.SetFinite((.*), 0)`
into a nicer `apd.Decimal.SetInt64($1)` which are equivalent.

Release note: None

49958: jobs: rename Resume to Unpause r=spaskob a=spaskob

Resume verb is already used to for the action of starting a job
after adoption on a node. Here instead Resume is the opposite
action of pausing a job.

Release note: none.

49961: opt: fix error caused by recursive CTE with zero rows on left side r=rytaft a=rytaft

Prior to this commit, a recursive CTE in which the cardinality of
the left side of the `UNION ALL` expression was zero would cause an
error in the statistics code, "estimated row count must be non-zero".
This was happening because the cardinality of the recursive CTE binding
props was set to be non-zero, but the row count, which came from the
left side expression, was not updated accordingly. The stats code only
allows the row count to be zero if the cardinality is also zero.

This commit fixes the problem by setting the row count of the binding
props to 1 if the estimated row count of the left side is less than 1.

Fixes cockroachdb#49911

Release note (bug fix): Fixed an internal planning error that occured
for recursive CTEs (WITH RECURSIVE expressions) in which the left side
of the UNION ALL query used in the CTE definition produced zero rows.

Co-authored-by: Jordan Lewis <jordanthelewis@gmail.com>
Co-authored-by: Rohan Yadav <rohany@alumni.cmu.edu>
Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: Spas Bojanov <spas@cockroachlabs.com>
Co-authored-by: Rebecca Taft <becca@cockroachlabs.com>

`sql/pgwire/testdata/encodings.json` is autogenerated using
`cmd/generate-binary/main.go`. Previously, a few tests were missing
from this file, which would have been lost the next time new tests were
added and `encodings.json` was autogenerated. This PR fixes that by
moving the tests to the auto-gen script.

Release note (none)

49682: sql: implement ConstructScan in the new factory r=yuzefovich a=yuzefovich

**sql: extract out planning of table readers into a separate method**

This commit extract most of the logic of `createTableReaders` into
a separate method that performs the actual physical planning of table
readers given already created `TableReaderSpec`s. This will allow for
reuse in the follow-up work. This commit additionally slightly refactors
a few other things for reuse.

Release note: None

**sql: implement ConstructPlan by new factory**

This commit extracts the logic of `execFactory.ConstructPlan` into the
function to be used by both factories. It also extracts out another
small helper.

Release note: None

**sql: implement ConstructScan in distsql spec factory**

This commit adds an implementation of
`distSQLSpecExecFactory.ConstructScan` which combines the logic that
performs `scanNode` creation of `execFactory.ConstructScan` and physical
planning of table readers of `DistSQLPlanner.createTableReaders`.
I tried to refactor the code so that there is not much duplication going
on.

Notably, simple projections, renders, and filters are not yet
implemented.

Closes: cockroachdb#47474.

Release note: None

**sql: enforce the assumption that `wantedColumns` is not nil for scans**

After the examining the code closer, I see that now in all code paths we
populate `scanColumnsConfig.wantedColumns` to ask for the specific
columns to be scanned. Previously, this map could be left as `nil` which
would mean that all columns of the table should be scanned. This was
used only by the heuristic planner which has been removed, so we can now
update the assumption of the scan columns config. This allows us to
clean up the comments a bit.

This commit also moves `makeColumnsConfig` function into the shared util
file because it is used by both factories.

Release note: None

Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>

49949: geo/geogfn: implement ST_Project r=otan a=hueypark

Fixes cockroachdb#48402

Release note (sql change): This PR implement adds the ST_Project({geography,float8,float8})

Co-authored-by: Jaewan Park <jaewan.huey.park@gmail.com>

Fixes cockroachdb#29021
Only create view dependencies on columns if the column is referenced in the view query.

Release note (sql change): Views now only create a dependency on a table's column if the
column is referenced in the view definition. Previously, all columns were added as a dependency
meaning if a table was referenced in a view, all columns regardless of if the column was actually
referenced would be added to the view's dependencies.

Fixes cockroachdb#49977

Parallel importer could get stuck  due to a race between emitted
import batches and checking for context cancellation (either due to an
unforeseen error, or due to explicit context cancallation).

Fix the race condition, and add tests verifying correct behavior.

Release notes (bug fix): correctly handle import cancellation and errors.

49872: sql: track fine-grained view dependencies r=rytaft a=RichardJCai

Fixes cockroachdb#29021
Only create view dependencies on columns if the column is referenced in the view query.

Note: this does not fix the dependencies of old views.

Release note (sql change): Views now only create a dependency on a table's column if the
column is referenced in the view definition. Previously, all columns were added as a dependency
meaning if a table was referenced in a view, all columns regardless of if the column was actually
referenced would be added to the view's dependencies.

49888: geo: fix GeoJSON and allow options in ST_AsGeoJSON r=sumeerbhola a=otan

There was a bug in GeoJSON where we used the high level name:"Feature"
field, as opposed to the inner Geometry. This has been fixed.

We also support more options for ST_AsGeoJSON now that these changes are
in twpayne/go-geom.

Resolves cockroachdb#48381.

Release note (sql change): Implement ST_AsGeoJSON with options to show
bbox and CRS information.

49931: opt: add rule to fold limits r=DrewKimball a=DrewKimball

Previously, there was no rule to fold a Limit on top of a Limit
when the outer limit value is smaller than the inner limit value.

This patch adds a rule to fold two Limits together when the
outer Limit has a smaller limit value than the inner Limit and
the inner ordering implies the outer ordering.

Release note (sql change): The optimizer can now fold two Limit
operators together.

Co-authored-by: richardjcai <caioftherichard@gmail.com>
Co-authored-by: Oliver Tan <otan@cockroachlabs.com>
Co-authored-by: Drew Kimball <drewk@cockroachlabs.com>

49979: importccl: Correctly handle errors and cancellations during import. r=miretskiy a=miretskiy

Fixes cockroachdb#49977

Parallel importer could get stuck  due to a race between emitted
import batches and checking for context cancellation (either due to an
unforeseen error, or due to explicit context cancallation).

Fix the race condition, and add tests verifying correct behavior.

Release notes (bug fix): correctly handle import cancellation and errors.

Co-authored-by: Yevgeniy Miretskiy <yevgeniy@cockroachlabs.com>

Allow and block aren't hurtful terms, and they also happen to be clearer
than white and black - they connote meaning instantly.

c.f.:

https://twitter.com/bradfitz/status/1269449722063773696?s=20
rails/rails#33677
https://gitlab.com/gitlab-org/gitlab/-/issues/7554
lagom/lagom#2532
https://bugs.chromium.org/p/chromium/issues/detail?id=981129

Release note: None

Release note (sql change): Introduce maxDecimalDigits arguments for
ST_AsText and ST_AsEWKT, which allow rounding of the decimal digits
output in the WKT representation.

49875: geo: handle maximum decimal digits for ST_AsText r=sumeerbhola a=otan

Resolves cockroachdb#48387.
Resolves cockroachdb#48885.

Release note (sql change): Introduce maxDecimalDigits arguments for
ST_AsText and ST_AsEWKT, which allow rounding of the decimal digits
output in the WKT representation.

Co-authored-by: Oliver Tan <otan@cockroachlabs.com>

49946: *: s/whitelist/allowlist/, s/blacklist/blocklist/ r=jordanlewis a=jordanlewis

Allow and block aren't hurtful terms, and they also happen to be clearer
than white and black - they connote meaning instantly.

Even though blacklist and whitelist don't have racist intent, they nevertheless
propagate an unconscious "white = allow", "black = deny" meaning. There's no
need to keep these terms - they're just terms. This is just one tiny thing we can
do to improve our industry's inclusivity.

c.f.:

https://twitter.com/bradfitz/status/1269449722063773696?s=20
rails/rails#33677
https://gitlab.com/gitlab-org/gitlab/-/issues/7554
lagom/lagom#2532
https://bugs.chromium.org/p/chromium/issues/detail?id=981129

Release note: None

Co-authored-by: Jordan Lewis <jordanthelewis@gmail.com>

This change refactors following components to
use CSS modules instead of styles defined as
global:
- Dropdown
- Search
- PageConfig

Release note: None

48012: ui: CSS modules for Statements filter section r=koorosh a=koorosh

Depends on cockroachdb#47606
Related to cockroachdb#47527

This change refactors following components to
use CSS modules instead of styles defined as
global:
- Dropdown
- Search
- PageConfig

Release note: None

Co-authored-by: Andrii Vorobiov <and.vorobiov@gmail.com>

This commit modifies structured.proto to duplicate the fields required for
leasing into each of the descriptor types. It does not yet adopt these fields.

This commit is part of a much larger change to wrap descriptor proto structs
used throughout the codebase. To contextualize the scope of this change, in an
earlier iteration this commit, instead of duplicating these fields, opted to
create a new message which was to be stored in the `Descriptor` type itself.

While that change brought with it some niceties, it also brought the requirement
to wrap the raw descriptor structs as they no longer carried all of their
relevant metadata. This duplication approach in many ways obviates the pressing
need for much of this PR but, nevertheless, we pursue it because it's part of
a broader trend we'd like to enforce. In many ways the scope of this PR has
become opportunistic rather than necessary.

Release note: None

For now, move the Descriptor interface down into sqlbase as part of the
effort to elimate sqlbase.DescriptorProto.

Release note: None

In preparation for merging that interface with DescriptorInterface.

Release note: None

This change is large but is largely mechanical. The basic idea is that we want
to stop passing around raw pointers to protobuf structs. Instead we'd like to
pass around higher-level wrappers which implement interfaces.

There's some room for debate about the precise nature of Mutable/Immutable
wrappers for descriptor types but over time, hopefully, we'll move descriptor
manipulation behind clear interfaces.

It's worth noting that this commit is a half-step. It began, somewhat
unfortunately, by introducing the TypeDescriptor wrappers and then goes
all the way to deleting the `DescriptorProto` interface and unexporting
`WrapDescriptor`.

Release note: None

…escriptor

Next we'll stop having the raw descriptor implement that interface.

Release note: None

This is a large but also mostly mechanical change. There's room for improvement
on the uniformity of it all but let's leave that be for now as I'm getting
tired and need to keep making progress. It'll be easier to clean it up later.

In particular, I've increasingly been feeling that an interface-based approach
to these descriptors would be worthwhile and that that would be easy to
accomplish for database but alas. That will have to be another big and
mechanical change.

Release note: None

This commit has GetDescriptorByID return an "unwrapped" descriptor and removes
`UnwrapDescriptor` function from sqlbase. This terminology is all crap given
the `UnwrapDescriptor` function returned a descriptor that is itself sort of
wrapped. Good news, this terminology is going away. There is a hodge-podge
of other unfortunate touch up in this commit.

Release note: None

Release note: None

This commit made a lot more sense when it removed those fields from the proto
and lifted them to a new message. It's still not a terrible commit so we'll
leave it. It should be handy when we transition descriptor interactions to
be through interfaces.

Release note: None

This commit, like others, is not nearly as urgent given we're keeping the
meta fields on the database descriptor.

This commit adopts the interface methods for accessing the ID and Name of
a DatabaseDescriptor. The reworking in this commit of previous changes also
in this PR is annoying, I'm sorry to the reviewers. I'm also increasingly
sensing the urgency of eschewing the concrete descriptor wrappers in favor
of interfaces but I'm not going to try to deal with that in this PR.

Release note: None

In anticipation of lifting methods to the wrapper types.

Release note: None

Release note: None

This commit makes more retrieval functions "unwrap" descriptors into
DescriptorInterface before returning them.

It somewhat unfortunately pushes that responsibility into the sql/resolver in
order to accomodate backups.

Release note: None

This commit is the ugly side of copying the leasing fields into each of the
descriptors. This leaves the `Descriptor.Table` method in place for now
as it forms defense in depth. We'll remove it eventually when this all
gets cleaned up.

Release note: None